Skip to content

chore: removed last klona#40257

Merged
vsvamsi1 merged 2 commits intoreleasefrom
test77
Apr 16, 2025
Merged

chore: removed last klona#40257
vsvamsi1 merged 2 commits intoreleasefrom
test77

Conversation

@vsvamsi1
Copy link
Contributor

@vsvamsi1 vsvamsi1 commented Apr 15, 2025

Description

  • Removed the final use of klona, which was a significant contributor to performance bottlenecks in our evaluation cycles.
  • Enhanced the dataTree reduction logic to improve the quality of diff updates. Previously, some diffs were not applied correctly, potentially leading to stale application states. This change should reduce related Sentry-reported issues.
  • In a configured customer app running on Windows, we observed approximately a 1-second improvement in LCP. Additionally, overall Web Worker scripting time is expected to decrease by about 29.6%. These optimizations primarily target update evaluation cycles and should lead to noticeably improved app responsiveness.

Automation

/ok-to-test tags="@tag.All"

🔍 Cypress test results

Tip

🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/14475541362
Commit: 9d0b751
Cypress dashboard.
Tags: @tag.All
Spec:


Tue, 15 Apr 2025 18:35:25 UTC

Communication

Should the DevRel and Marketing teams inform users about this change?

  • Yes
  • No

Summary by CodeRabbit

Summary by CodeRabbit

  • Bug Fixes

    • Improved error handling for evaluation errors, with enhanced logging and reporting for specific error types.
  • Tests

    • Added comprehensive test cases covering data tree type changes, update cycles, error handling during state updates, and evaluation flow accuracy.
  • Refactor

    • Refactored data tree diffing, update logic, and evaluation state management for better modularity, error resilience, and control flow during evaluation cycles.
  • New Features

    • Introduced granular tracking, serialization, and application of data tree updates to improve performance and reliability.

@vsvamsi1 vsvamsi1 added the ok-to-test Required label for CI label Apr 15, 2025
@vsvamsi1 vsvamsi1 self-assigned this Apr 15, 2025
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Apr 15, 2025

Walkthrough

This set of changes introduces new error handling for UPDATE_DATA_TREE_ERROR within the evaluation error handler, updates the EvalErrorTypes enum, and adds comprehensive tests for data tree diffing and update logic. The evaluation and diffing helpers are refactored to modularize and enhance the process of generating, serializing, and applying updates between data tree states, with explicit tracking of update cycles and improved error capture. Test suites are expanded to cover new scenarios, and the control flow for data tree evaluation and update application is adjusted to use the new helpers and error types.

Changes

File(s) Change Summary
app/client/src/sagas/EvalErrorHandler.ts Added a new case for UPDATE_DATA_TREE_ERROR in the error handler, reporting to Sentry and logging locally.
app/client/src/utils/DynamicBindingUtils.ts Extended EvalErrorTypes enum with UPDATE_DATA_TREE_ERROR.
app/client/src/workers/Evaluation/helpers.ts Refactored diffing logic: replaced getReducedDataTree with getReducedDataTrees, modularized update serialization and application, added error handling for update cycles, and introduced new helpers (updatePrevState, updateEvalProps).
app/client/src/workers/Evaluation/handlers/evalTree.ts Refactored data tree construction to use updateEvalProps, added isUpdateCycle flag, adjusted update generation and previous state logic.
app/client/src/workers/Evaluation/evalTreeWithChanges.ts Updated to use new helpers for data tree preparation and update generation; adjusted assignment and type assertions.
app/client/src/workers/Evaluation/helpers.test.ts Added tests for updatePrevState, covering normal, update cycle, and error scenarios.
app/client/src/workers/Evaluation/evalTreeWithChanges.test.ts Updated test setup and assertions to reflect new diff semantics and helper usage.
app/client/src/workers/Evaluation/tests/generateOpimisedUpdates.test.ts Added extensive tests for type change diffs and data tree reduction; refined assertions for update kinds.

Sequence Diagram(s)

sequenceDiagram
  participant UI
  participant Saga
  participant EvalWorker
  participant Sentry

  UI->>Saga: Trigger evaluation
  Saga->>EvalWorker: Evaluate data tree
  EvalWorker->>EvalWorker: Generate diffs & updates (using helpers)
  alt Error: UPDATE_DATA_TREE_ERROR
    EvalWorker->>Saga: Throw UPDATE_DATA_TREE_ERROR
    Saga->>Sentry: Sentry.captureMessage(error)
    Saga->>Saga: log.error("Evaluation Error: ...")
  else Success
    EvalWorker->>Saga: Return evaluation result & updates
  end
  Saga->>UI: Send evaluation response
Loading

Possibly related PRs

  • appsmithorg/appsmith#40170: Adds a new error type UPDATE_DATA_TREE_ERROR handling case to the existing evalErrorHandler function, directly related to the enhanced error logging and Sentry reporting in this PR.

Suggested reviewers

  • dvj1988
  • rajatagrawal

Poem

When data trees twist and change their form,
New helpers and tests ensure we weather the storm.
Errors now logged, Sentry gets the call,
While update cycles march, never to stall.
With diffs and patches, the code stands tall—
Evaluation’s robust, thanks to one and all!
🌳✨


📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between 2b2e5b7 and 9d0b751.

📒 Files selected for processing (1)
  • app/client/src/workers/Evaluation/handlers/evalTree.ts (6 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/client/src/workers/Evaluation/handlers/evalTree.ts
⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: perform-test / rts-build / build
  • GitHub Check: perform-test / server-build / server-unit-tests
  • GitHub Check: client-prettier / prettier-check
  • GitHub Check: client-check-cyclic-deps / check-cyclic-dependencies
  • GitHub Check: client-build / client-build

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added the skip-changelog Adding this label to a PR prevents it from being listed in the changelog label Apr 15, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (3)
app/client/src/workers/Evaluation/evalTreeWithChanges.ts (1)

156-162: Added support for differentiating update cycles.

The new boolean parameter (true in this case) indicates that this is an update cycle, enabling optimized diff generation.

I recommend adding a brief comment explaining what this boolean parameter controls to improve code clarity for future developers.

 const updates = generateOptimisedUpdatesAndSetPrevState(
   dataTree,
   dataTreeEvaluator,
   affectedNodePaths,
   additionalUpdates,
-  true,
+  true, // isUpdateCycle: true indicates this is an update to existing data tree, not initial creation
 );
app/client/src/workers/Evaluation/helpers.ts (2)

528-560: Error handling for update application is a good addition

The updatePrevState function includes proper error handling for update application, which helps prevent cascading failures. However, consider logging these errors more extensively for debugging purposes.

- dataTreeEvaluator.errors.push({
-   type: EvalErrorTypes.UPDATE_DATA_TREE_ERROR,
-   message: error.message,
- });
+ dataTreeEvaluator.errors.push({
+   type: EvalErrorTypes.UPDATE_DATA_TREE_ERROR,
+   message: error.message,
+   context: {
+     path: update.path,
+     updateKind: update.kind
+   }
+ });

508-508: Consider using nullish coalescing operator

For better handling of falsy values, consider using the nullish coalescing operator (??) instead of logical OR (||).

- dataTreeEvaluator?.getPrevState() || {},
+ dataTreeEvaluator?.getPrevState() ?? {},
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b39d21e and 2b2e5b7.

📒 Files selected for processing (8)
  • app/client/src/sagas/EvalErrorHandler.ts (1 hunks)
  • app/client/src/utils/DynamicBindingUtils.ts (1 hunks)
  • app/client/src/workers/Evaluation/__tests__/generateOpimisedUpdates.test.ts (3 hunks)
  • app/client/src/workers/Evaluation/evalTreeWithChanges.test.ts (6 hunks)
  • app/client/src/workers/Evaluation/evalTreeWithChanges.ts (4 hunks)
  • app/client/src/workers/Evaluation/handlers/evalTree.ts (7 hunks)
  • app/client/src/workers/Evaluation/helpers.test.ts (2 hunks)
  • app/client/src/workers/Evaluation/helpers.ts (6 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (3)
app/client/src/workers/Evaluation/evalTreeWithChanges.test.ts (1)
app/client/src/workers/Evaluation/helpers.ts (1)
  • updateEvalProps (562-580)
app/client/src/workers/Evaluation/helpers.test.ts (2)
app/client/src/workers/Evaluation/handlers/evalTree.ts (1)
  • dataTreeEvaluator (40-40)
app/client/src/workers/Evaluation/helpers.ts (1)
  • updatePrevState (528-560)
app/client/src/workers/Evaluation/handlers/evalTree.ts (1)
app/client/src/workers/Evaluation/helpers.ts (1)
  • updateEvalProps (562-580)
⏰ Context from checks skipped due to timeout of 90000ms (8)
  • GitHub Check: perform-test / client-build / client-build
  • GitHub Check: perform-test / rts-build / build
  • GitHub Check: perform-test / server-build / server-unit-tests
  • GitHub Check: client-unit-tests / client-unit-tests
  • GitHub Check: client-lint / client-lint
  • GitHub Check: client-check-cyclic-deps / check-cyclic-dependencies
  • GitHub Check: client-build / client-build
  • GitHub Check: client-prettier / prettier-check
🔇 Additional comments (26)
app/client/src/utils/DynamicBindingUtils.ts (1)

163-163: Added new error type for data tree update failures.

The new UPDATE_DATA_TREE_ERROR enum value will help track and handle errors specifically occurring during data tree updates.

app/client/src/sagas/EvalErrorHandler.ts (1)

306-314: Appropriate error handling implementation for the new error type.

The implementation follows the established pattern in the file - capturing the message in Sentry and logging locally with error details. This maintains consistency with other error handlers.

app/client/src/workers/Evaluation/evalTreeWithChanges.test.ts (4)

8-8: Updated imports for modified evaluation flow.

The imports of klona and updateEvalProps align with the changes in the evaluation flow that now uses these for data tree cloning and property updates.

Also applies to: 14-14


206-209: Test setup refactored to use new evaluation helpers.

The setup now leverages updateEvalProps to update evaluation properties and uses klona for deep cloning, matching the new evaluation flow implementation.


379-387: Updated diff kind expectations to match new diff semantics.

The test now correctly expects kind "E" (edit) instead of "N" (new), aligning with the refined diff semantics implemented in the evaluation helpers.


531-535: Updated test expectations for precise update generation.

The test now verifies exact arrays of edits including dependent widget updates, which reflects the more precise update generation in the evaluation helpers.

app/client/src/workers/Evaluation/evalTreeWithChanges.ts (3)

15-16: Updated imports for new evaluation flow.

Importing updateEvalProps and additional types to support the refactored evaluation process.

Also applies to: 19-20


133-133: Replaced manual entity config handling with helper function.

Using updateEvalProps instead of (likely) makeEntityConfigsAsObjProperties simplifies the code and aligns with the modularized evaluation approach.


147-147: Improved type safety with explicit type assertion.

The explicit type assertion for additionalUpdates makes the code more type-safe and self-documenting.

app/client/src/workers/Evaluation/helpers.test.ts (5)

1-3: Imports look clean. No concerns here.


5-96: Comprehensive tests for stringifyFnsInObject. Good coverage for nested and array scenarios.


98-129: Accurate test for non-update cycle scenario. Validates setPrevState usage correctly.


130-174: Good coverage of serialized updates on update cycles. Verifies partial state updates thoroughly.


175-221: Solid error-handling test. Ensures errors are pushed to dataTreeEvaluator.errors as expected.

app/client/src/workers/Evaluation/handlers/evalTree.ts (5)

22-22: New import updateEvalProps. This utility looks well-integrated with existing helpers.


95-96: Introduction of isUpdateCycle flag. Straightforward approach to track update cycles.


136-136: Usage of updateEvalProps. Setting dataTree from evaluator ensures correct property updates each cycle.

Also applies to: 188-188, 246-246


192-192: Adjusting isUpdateCycle mid-flow. The on/off pattern is clear and synced to the evaluation cycle.

Also applies to: 336-336


327-329: Passing isUpdateCycle to generateOptimisedUpdatesAndSetPrevState. Maintains consistent logic for partial diffs.

app/client/src/workers/Evaluation/__tests__/generateOpimisedUpdates.test.ts (4)

13-18: Added type imports & getReducedDataTrees. These imports align well with the new tests.


412-419: Refined checks for removed function properties. Ensures the correct diffs are generated and applied.


675-954: Robust “type change tests.” Thorough scenarios covering array-to-object, object-to-array transformations, and partial property diffs.


956-1114: Focused tests for getReducedDataTrees. Properly validates selective reduction and special handling of __evaluation__ properties.

app/client/src/workers/Evaluation/helpers.ts (3)

185-266: Improved DataTree reduction with ancestor path handling

The updated getReducedDataTrees function is a significant improvement over the previous implementation. It now:

  1. Handles both old and new trees simultaneously
  2. Preserves error information in both trees
  3. Intelligently walks up the path hierarchy to find defined ancestors
  4. Optimizes the resulting trees to contain only the necessary data

This should improve the performance and accuracy of diffs.


276-281: Good implementation of reduced trees in diff generation

Using the new getReducedDataTrees function here properly supports the implementation of the optimized diffing approach. This will reduce the amount of data being compared, making the diff generation more efficient.


499-526: Good refactoring of update generation into a named function

Converting the anonymous function to a named function with explicit parameters improves code readability and maintainability. The added isUpdateCycle flag provides better control over state management.


import { create } from "mutative";
export const fn_keys: string = "__fn_keys__";
import { klona } from "klona/json";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

The PR objective mentions removing klona but it's still imported and used

This PR's objective was to remove the last usage of "klona," but it's still imported here and used in the updateEvalProps function (line 574). Consider completing the klona removal by using an alternative clone mechanism.


🏁 Script executed:

#!/bin/bash
# Check if klona is used elsewhere in the codebase
rg -i "klona" --type=ts --type=js | grep -v "package.json"

Length of output: 25609


Critical: Complete Removal of klona from helpers.ts

Our investigation confirms that the PR’s goal to remove klona has not been fully achieved. In app/client/src/workers/Evaluation/helpers.ts, klona is still being imported from "klona/json" (line 12) and used within the updateEvalProps function (line 574). Please replace these references with the approved alternative clone mechanism to align with the PR objective.

Comment on lines +562 to +580
export function updateEvalProps(dataTreeEvaluator: DataTreeEvaluator) {
if (!dataTreeEvaluator) return null;

const evalProps = dataTreeEvaluator.evalProps;

return create(dataTreeEvaluator.evalTree, (draft) => {
for (const [entityName, entityEvalProps] of Object.entries(evalProps)) {
if (!entityEvalProps.__evaluation__) continue;

set(
draft[entityName],
"__evaluation__",
klona({
errors: entityEvalProps.__evaluation__.errors,
}),
);
}
});
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider alternative to klona for deep cloning

The function correctly updates evaluation properties, but uses klona for cloning which contradicts the PR objective of removing klona usage.

- klona({
-   errors: entityEvalProps.__evaluation__.errors,
- }),
+ {
+   errors: JSON.parse(JSON.stringify(entityEvalProps.__evaluation__.errors)),
+ },

Alternatively, you could use the immutable approach with create from mutative:

- klona({
-   errors: entityEvalProps.__evaluation__.errors,
- }),
+ create({}, (draft) => {
+   draft.errors = entityEvalProps.__evaluation__.errors;
+ }),
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
export function updateEvalProps(dataTreeEvaluator: DataTreeEvaluator) {
if (!dataTreeEvaluator) return null;
const evalProps = dataTreeEvaluator.evalProps;
return create(dataTreeEvaluator.evalTree, (draft) => {
for (const [entityName, entityEvalProps] of Object.entries(evalProps)) {
if (!entityEvalProps.__evaluation__) continue;
set(
draft[entityName],
"__evaluation__",
klona({
errors: entityEvalProps.__evaluation__.errors,
}),
);
}
});
}
export function updateEvalProps(dataTreeEvaluator: DataTreeEvaluator) {
if (!dataTreeEvaluator) return null;
const evalProps = dataTreeEvaluator.evalProps;
return create(dataTreeEvaluator.evalTree, (draft) => {
for (const [entityName, entityEvalProps] of Object.entries(evalProps)) {
if (!entityEvalProps.__evaluation__) continue;
set(
draft[entityName],
"__evaluation__",
{
errors: JSON.parse(JSON.stringify(entityEvalProps.__evaluation__.errors)),
},
);
}
});
}

// serialise bigInt values and convert the updates to a string over here to minismise the cost of transfer
// to the main thread. In the main thread parse this object there.
return { serialisedUpdates: serialiseToBigInt(removedLhs) };
return { serialisedUpdates: serialiseToBigInt(removedLhs), updates };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Return type extended to include raw updates

The function now returns both serialized updates and the raw updates array. Ensure all call sites have been updated to handle this change.


🏁 Script executed:

#!/bin/bash
# Find all places where generateSerialisedUpdates is called
# and check if they handle the new return value
rg "generateSerialisedUpdates" --type=ts --type=js -A 3 -B 1

Length of output: 9247


Return type extended – Call sites must account for raw updates

The function now returns an object with both a serialisedUpdates property and a new updates array. However, our search indicates that many call sites (notably within app/client/src/workers/Evaluation/__tests__/generateOpimisedUpdates.test.ts) still only destructure and use serialisedUpdates (and in one case, error). Please update these call sites to either destructure the new updates field where needed or explicitly confirm that ignoring it is acceptable given the intended business logic.

  • Action items:
    • Review usages of generateSerialisedUpdates in app/client/src/workers/Evaluation/helpers.ts and its tests.
    • Ensure that where the raw updates array is required, the caller properly extracts and handles the updates property.

@vsvamsi1 vsvamsi1 merged commit f9d2e2f into release Apr 16, 2025
89 checks passed
@vsvamsi1 vsvamsi1 deleted the test77 branch April 16, 2025 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Required label for CI skip-changelog Adding this label to a PR prevents it from being listed in the changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants